Field of the Invention
The present invention relates to an image processing apparatus, image processing method, and an image capturing apparatus, and in particular relates to an image capturing apparatus having a camera shake correction function and a refocus function, and an image processing apparatus and image processing method that processes an image signal obtained from the image capturing apparatus.
Description of the Related Art
Conventionally, a digital camera having a camera shake correction function has been proposed. With a digital camera having a camera shake correction function, the camera shake correction function is realized by changing the attitude of an optical member and/or an image sensor in a desired direction according to a detected camera shake amount. With the method of changing the attitude of an optical member, it is possible to widen the angle in which correction is possible by changing multiple optical members in respective independent directions.
Japanese Patent Laid-Open No. 2009-258389 discloses a method in which a frontward first movable lens barrel that supports a first optical member and a rearward second movable lens barrel that supports a second optical member are arranged with a fixing member interposed therebetween, and thereby the movable lens barrels are driven independently of each other so as to correct camera shake. Also, Japanese Patent No. 3003370 and Japanese Patent Laid-Open No. 2008-134329 disclose a method of correcting camera shake by driving an optical member such that an arc is traced with a point on an optical axis serving as the center of rotation.
Further, Japanese Patent Laid-Open No. 2008-134329 discloses a method of realizing the driving of an optical member such that an arc is traced with a point on an optical axis serving as the center of rotation by arranging a plurality of spheres between a holding frame of a convex spherical lens and a holding member of a concave spherical lens, and moving the convex spherical lens.
On the other hand, by arranging a microlens array with a ratio of one microlens for a plurality of pixels on a front surface of an image sensor, it is possible to acquire not only a two-dimensional intensity distribution of light, but also information on the entrance direction of light rays that enter the image sensor, and to obtain three-dimensional information on the subject space. A camera capable of obtaining this kind of three-dimensional information on the subject space is called a light-field camera. Moreover, the three-dimensional information on the subject space is called light-field data, and by acquiring the light-field data and performing image reconstruction after shooting, it is possible to perform image processing known as refocusing, such as changing the focus position of the image, changing the shooting viewpoint, and adjusting the depth of field.
With this kind of light-field camera, a plenoptic method is widely known. With the plenoptic method, divided photoelectric conversion elements (PD) for image capture are arranged two-dimensionally below microlenses in a microlens array, and focus lenses included in an optical system serve as exit pupils for the microlenses. In an image capturing apparatus with this kind of configuration, it is known that signals obtained from multiple PDs existing below the microlenses include multiple pieces of light ray information from the subject. Multiple two-dimensional images, which are each formed using, among the signals obtained from the group of PDs located below the microlenses, only signals obtained from PDs that exist at the same location with respect to each microlens using the light ray information, have parallax with respect to each other, unlike normal two-dimensional images. By compositing the two-dimensional images with such parallax, it is possible to virtually move the focus plane of the image (see U.S. Pat. No. 8,358,367).
However, even if an image stabilization optical system is driven in methods as disclosed in the Japanese Patent No. 3003370 and Japanese Patent Laid-Open No. 2008-134329, it is difficult to keep an image plane and an image pickup surface in parallel in every state of the image stabilization optical system.
Further, in the image stabilization optical system described in Japanese Patent Laid-Open No. 2008-134329, since an optical member is designed so as to optimally keep an image plane and an image pickup surface in parallel at a given focal length, it is difficult to keep the image plane and the image pickup surface in parallel depending on the focal length of a zoom lens.
As described above, when an image stabilization optical system is used, there may be a case where uneven blurring occurs in a shot image because an image plane and an image pickup surface cannot be kept in parallel. Especially, if uneven blurring occurs in each frame while shooting a moving image, a problem occurs in that the quality of the shot moving image is significantly reduced.